Parallel Gibbs Sampling: From Colored Fields to Thin Junction Trees
نویسندگان
چکیده
We explore the task of constructing a parallel Gibbs sampler, to both improve mixing and the exploration of high likelihood states. Recent work in parallel Gibbs sampling has focused on update schedules which do not guarantee convergence to the intended stationary distribution. In this work, we propose two methods to construct parallel Gibbs samplers guaranteed to draw from the targeted distribution. The first method, called the Chromatic sampler, uses graph coloring to construct a direct parallelization of the classic sequential scan Gibbs sampler. In the case of 2-colorable models we relate the Chromatic sampler to the Synchronous Gibbs sampler (which draws all variables simultaneously in parallel), and reveal new ergodic properties of Synchronous Gibbs chains. Our second method, the Splash sampler, is a complementary strategy which can be used when the variables are tightly coupled. This constructs and samples multiple blocks in parallel, using a novel locking protocol and an iterative junction tree generation algorithm. We further improve the Splash sampler through adaptive tree construction. We demonstrate the benefits of our two sampling algorithms on large synthetic and real-world models using a 32 processor multi-core system.
منابع مشابه
HUGS: Combining Exact Inference and Gibbs Sampling in junction Trees
Dawid, Kjrerulff & Lauritzen (1994) provided a preliminary description of a hybrid between Monte-Carlo sampling methods and exact lo cal computations in junction trees. Utiliz ing the strengths of both methods, such hy brid inference methods has the potential of expanding the class of problems which can be solved under bounded resources as well as solving problems which otherwise resist ex ...
متن کاملGibbs Sampling Segmentation of Parallel Dependency Trees for Tree-Based Machine Translation
We present a work in progress aimed at extracting translation pairs of source and target dependency treelets to be used in a dependency-basedmachine translation system. We introduce a novel unsupervised method for parallel tree segmentation based on Gibbs sampling. Using the data from a Czech-English parallel treebank, we show that the procedure converges to a dictionary containing reasonably s...
متن کاملFast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks and Factor Graphs
A fundamental task in machine learning and related fields is to perform inference on probabilistic graphical models. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has historically been too expensive for large models. In this paper, w...
متن کاملFrom Fields to Trees
We present new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the ...
متن کاملFast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks
A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has historically been too expensive for large models. In this paper, we present an ...
متن کامل